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ABSTRACT 


This report docxunents and summarizes Che accomplishments 
over Che past year in two areas: (1) development of Landsat classi- 

ficCion accuracy assessment techniques, and (2) development of a 
computerized system for assessing wildlife habitat from land cover 
maps. This report includes a literature review on accuracy 
assessment techniques, a complete explanation for the techniques 
developed under both projects, including example analyses and 
listings of Che computer programs. 

A summary of the presentations and discussions at Che 
National Worlclng Conference on Landsat Classification Accuracy is 
Included. Also, two symposium papers which have been published 


on the results of this project are included as appendices. 





Introduction 


Many studies have been conducted to determine the usefulness 


of LANDSAT data for mapping land cover. However, very little research has 


been done to determine the degree of success (i.e 


accuracy) in doing this 


A recent literature review by Mead (1977) indicated that 


. . .more work is needed to develop reliable techniques for 
estimating classification accuracies. A means of comparing 
the accuracies (i.e., to compare classification matrices) 
obtained in different areas on different dates, or estimated 
by different techniques is needed. Such techniques should permit the 
investigator to test hypotheses that at specified level of 
confidence the accuracies from several areas, dates, etc. 
are not different j 


Mead (1977) continues by suggesting "Future studies might consider 


iterative proportional fitting of the classification matrices as a 


means of doing this." (Bishop £t al . 1975) 


The apparent absence of quantitative methods for comparing class! 


fication accuracy is certainly a stumbling block that must be overcome 


The effects of imaging date, spectral band combination, classification 


classification accuracy must be studied. Therefore, the following study 


was proposed with these objectives 


within classification error matrices 




i 
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2. To develop hlerarchal models for testing the significance 
of several factors (e.g., Image date, classification 
algorithm, the analyst, etc.) on the resulting classifi- 
cation accuracy. 


3. To test the above techniques and determine their usefulness || 

with actual data for classification accuracy. P 

1.2 Justification 

Research will undoubtedly continue toward development of a system 
for classification of land cover from digitally recorded Landsat Imagery. 

Such research efforts will In part be measured by Improvements In the 
classification accuracies achieved. Therefore scientists will need ways 
of assessing the accuracy. Also the accuracy of the final maps produced 
must be verified before they are distributed to users. Once standards 
are established, rigorous statistical procedures will be needed to maintain 
the quality of the maps. Therefore, it can be seen that accuracy assess- 
ment techniques will be needed in both the research and operational 
environments . 

1 . 3 State of the Art of Landsat Classification Accuracy Assessment 
Landsat, like any other remote sensing system, is only as good 


1 

■;( 



as our ability to evaluate it. The need for techniques to assess the 
accuracy of the Landsat sensor systems cannot be understated. As 
Freese (1960) states, "testing the accuracy of some measurement against 
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an accepted standard requires a statement of the accuracy required, a 
measure of the accuracy attained, and an objective method of deciding 
whether the accuracy attained Is equal to the accuracy required". If 
there are no methods for measuring the accuracy attained with a certain 
sensor system, then there will be no way to make comparisons between 
systems to determine which Is better. 

If Landsat Is ever to become an operational system, then 
evaluation and accuracy assessment techniques must be developed to 
show where such sensor systems give more adequate results than con- 
ventional methods. These assessment techniques must then be applied 
to specific applications. For example, "the usefulness of satellite 
Imagery for forestry depends on the extent to which forest data can 
be recorded by a remote sensing system from satellite altitudes, pro- 
cessed by an Image Interpretation system, and used In forest mapping 
and inventories" (Kalensky and Scherk, 1975). 

1.31 Accuracy Assessment Techniques 


There have been very few studies done on accuracy of Landsat 
classification. Most of the early assessments were done as an "after 
thought" without much consideration given to the statistical methods 
used. These studies, such as the one done by Kalensky and Scherk (1975), 
usually dealt only with training set accuracy. The use of training sets 
as well as other possible areas to be assessed will be discussed later. 
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A review of the current assessment techniques are necessary before 
any of the applications of these techniques can be understood. 

The most common way to describe the accuracy of a Landsat image 
is in the form of an error matrix (e.g., Todd ^ , 1980; Mead and 

Meyer, 1977; Hoffer, 1975). An error matrix is a square array of 
numbers set out In rows and columns which express the number of pixels 
assigned as a particular land cover type relative to the actual land 
cover as verified in the field or from photos. The columns usually 
represent the ground truth and the rows indicate the computer assigned 
land cover category. This form of expressing accuracy 
as an error matrix allows for an effective way to evaluate 
both errors of Inclusion (commission errors) and errors of exclusion 
(omission errors) present in the classification. Also, the error matrix 
allows the analyst to determine the performance for individual categories 
as well as for the overall classification (Hoffer and Fleming, 1978). 

In the ideal situation, all the nou-major diagonal elements of the 
error matrix would be zero, indicating that no pixel had been misclassl- 
fied (Lillesand and Kiefer, 1979). 

There are two basic types of accuracy assessments. They are site 
specific accuracy and non-site specific accuracy. All the methods 
described to assess accuracy can be applied to either type. Non-site 
specific accuracy is less useful than site specific accuracy. Meyer 
et al . (1975) used a non-site specific accuracy assessment to evaluate 


r 




classification of Landsat Imagery In Southeastern Montana. Total area 


acraages were calculated for each Informational class. There were no 


tests made for positional accuracy Cslte specific). Just relative total 


acreages. Meyer found the estimate of the relative proportion of each 


cover type compared favorably with the ground truth (l.e., actual acres 


of each land cover category). However, he also noticed that omission 


and commission errors were very obvious and that the overall positional 


accuracy of the cover types within the areas studied was poor. 


This example points out the major disadvantages of a non-site 


specific accuracy assessment. If only total acreage estimates are needed. 


then this method may apply. However, the natural resource manager Is 


usually Interested In the location as well as the acreage of a certain 


land cover category. If this Is the case. It Is obvious that non-site 


specific accuracy assessment Is not adequate. 


Site specific accuracy, on the other hand. Is a measure of how 


well the computer (classification algorithm) classifies each pixel with 


respect to the ground truth. It Is a more meaningful representation of 


the accuracy of the classification. The analyst can see which categories 


are easily Identifiable and which are being confused. Although Lyon (1979) 


used site specific accuracy assessment, he Includes no error matrices In 


his paper. Instead, he gives just one number as a measure of the accuracy. 


This Is a common problem throughout the literature. Without error matrices. 


- iWWitl. liilUT 
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the reader has little chance of understanding how an accuracy figure 
was determined. The reader also loses the knowledge of which cate- 
gories were easily Identified and which were difficult. 

Once the error matrix has been generated, a very simple procedure 
can be used to determine the overall accuracy. Since all the values on 
the major diagonal represent those pixels that have been correctly 
classified. If one adds up the major diagonal and divides this number 
by the total number of pixels classified, one will obtain the overall 
accuracy of that error matrix. This Is the most common use of the error 
matrix In accuracy assessment. 

In recent years, some new techniques have been developed to assess 
classification accuracy. Among these new methods are analysis of vari- 
ance techniques, regression analysis techniques, and discrete multi- 
variate analysis techniques. Each of these methods has certain assumpt- 
ions that must be met^before the technique can be used for assessing 
classification accuracy. If these assumptions are not met, the technique 
loses Its power. 

The data used In classification accuracy assessment Is of the 
discrete type. Discrete data, as opposed to continuous data, may take 
on only a limited number of distinct values (Snedecor and Cochran, 1976). 
In analysis of variance, the data must be normally distributed In order 
to meet the assumptions of the technique. Since discrete data Is not 
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normally distributed, It would seem that ANOVA Is not a good technique 
for accuracy assessment. However, Rosenfleld (1980) has proposed the 
use of the logit transformation or the arcsine transformation as described 
by Snedecor and Cochran (1976) to transform the data Into an approximately | 

normal distribution. Rosenfleld states, "the statistically Interpreted 
results of the weighted adjustment agree fairly well with what might be 
technologically expected, and are therefore judged technically accept- 
able”. After the transformation Is applied to the data, the analysis 

i 

of variance can be run. From the resulting ANOVA table, multiple range | 

tests are applied to population means found to be significantly different 

I 

(Rosenfleld, 1980). Analysis of variance Is a powerful statistical tool. 

However, other techniques that do not require so much data manipulation 
should also be tested. Rosenfleld (1978) agrees, "this does not mean 
that they (ANOVA) are the best; however, the tools available should be 
used until something better comes along". 

Regression analysis Is another way of visually representing 
accuracy. In this case the ground truth (l.e., actual land cover) Is 
the Independent variable, X, and the computer classification Is the 
dependent variable, 1. If the computer Is completely correct In Its 
classification, then all the points will lie on a forty five degree line. 

Hore likely, the points will be spread out from this line, llie value of 
the correlation coefficient can then be used to get an Idea of the 
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relative agreement between the ground truth and the computer classi- 
fication. Regression analysis has not been widely used in the litera- 
ture and therefore no more will be said about it. 

1.32 Sampling Techniques 

The need to use more than just training areas for accuracy 
assessment has already been discussed. However, one could not afford 
nor desire to assess the entire scene. Instead, a representative sample 
should be chosen and assessed as the accuracy for the entire scene. 
Sampling allows not only the calculation of a number that represents 
the accuracy of the classification, but also allows for a confidence 
interval to be placed around that number. 

Ginevan (1979) states three criteria that should be satisfied in 
any sampling scheme. -These criteria are: (1) the sampling scheme should 

have a low probability of accepting a map of low accuracy, (2) the sampl- 
ing scheme should have a high probability of accepting a map of high 
accuracy, and (3) the sampling scheme should have a minimum number, N, 
of ground truth samples. Many researchers (Hay, 1979; Ginevan, 1979; and 
Genderen and Lock, 1977) agree that stratified random sampling is the 
best sampling scheme to use. Rhode (1978) proposes other schemes 
including cluster-stratified sampling and two phase sampling. Ho matter 
which sampling scheme is used, it should be chosen so to obtain the 
mcucimum Information with the minimum amount of work. This involves 
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considering many variables such as terrain, Image Identifiable 
loatlons, and variability of land cover categories. 

It should also be noted that errors arise In classification 
from other sources besides the sampling scheme chosen. Problems arise 
In radiometric correction and geometric rectification. Also, the time 
Interval between when the Imagery Is attained and when the field check- 
ing Is done may cause differences In land cover category. It must also 
be realized that Just because the classification of a category seems 
perfect, this does not always mean that the method Is error free. The 
result may occur purely by chance because of the sampling design. "This 
fact Is seldom appreciated by many Image Interpreters when checking the 
accuracy results of their remote sensing land use survey (Genderen and 
Lock, 1977). 

Finally, no matter which sampling scheme Is chosen, a sample 
size must be determined. This situation Is described by Glnevan (1979), 
'The sampling problem as defined here Is the determination of the optimal 
number, N, of ground truth samples and an allowable number, X, of mls- 
classlflcatlons of these samples." Once these have been determined, the 
results of Image Interpretation are checked against the N ground truth 
samples and the map Is accepted as accurate If X or fewer of the ground 
truth samples are mlsclasslfled. The optimum number of samples, N, to be 
taken has met with widespread disagreement throughout the literature 
(Todd et al. . 1980; Hay, 1979; Genderen et al. , 1978; Genderen and Lock, 
1977; and Hord and Brooner, 1976). Each researcher seems to have his 
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own Ideas about sample size determination and It Is obvious that a 
great deal more research Is needed In this area. 

1.33 National Data Base for Error Matrices 


Letters were sent out to potential sources of error matrices 
asking that any matrices they had be sent to us for Inclusion In a 
National Data Base for Error Matrices. An Information questionnaire was 
sent along with each request for data. This questionnaire contained 
questions about the location of the area arialyzed, the analyst, the 
algorithm, and the date the data were taken. j 

All error matrices that we have received have been compiled 
along with their corresponding pertinent Information and placed on a 
computer tape. This data are available for distribution to other users 
upon request. A listing of the sources of error matrices can be 


found In Appendix I. 






2. 0 Statistical Methods 

2. 1 Methods of Comparing Similarity Matrices 

Two methods were used to compare two-dimensional matrices 
representing ground classification versus machine classification 
from different methods. In the first method of comparison the 
cell entries In each matrix are successively balanced until the 
sum of each of the matrix margins Is one. The entries In the 
matrix then represent a normalized percentage of the total 
observations occurring In each matrix cell. Within an Individual 
matrix these percentages can be used to examine omission and 
commission errors. Classification errors between two or more 
machine classification methods can be evaluated by comparing the 
percentages In corresponding cells In each matrix. Matrices with 
differing numbers of observations can be compared since the entries 
In each matrix are transformed to percentages. 

The second method of comparison was a measure of agreement 
for two-dimensional square matrices presented by Bishop al. (1975). 
This measure, K, Is calculated as the difference between the actual 
agreement and chance agreement between two classification methods. 

In this application the two methods are ground classification and 


machine classification. The measure Is calculated as 



totals of row 1 and column 1 


and to perform tests for equality of K between two matrices 


The two methods described above can be used together. Method 


two, K, will Indicate whether two matrices exhibit the same degree 


method one can be used to determine In which particular category 


The Influence of factors such as season of Imaging, film type 


categorical data analysis (Bishop et al. , 1975). Using this analysis 


factor or combination of factors can be assessed. 
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Categorical data analysis requires only Chat each factor 
being examined for Influence on accuracy can be assigned to an 
unambiguous category within each factor. These categories may 
be normative, ordinal, or Interval. The result of data 
collection Is a multidimensional matrix with each factor. Including 
ground and machine classification, serving as a dimension of the 
matrix. 

This method of analysis avoids the more restrictive 
assumptions Inherent In alternative analysis methods such as 
multivariate regression or analysis of variance. No normality 
assumption Is necessary, no factors need be considered as con- 
tinuous, and Interpretation of many dummy variables Is avoided. 
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3.0 Sample Data Analyses 

3.1 MARGFIT Analysis 

As previously discussed, the FORTRAN computer program MARGFIT 
Csee Appendix II) Implements a normalization procedure which standardizes 
each error matrix for purposes of comparison. The accuracy of the 
classification can then be represented as a normalized overall perform- 
ance. This value Is calculated the same way as in overall performance 
(i.e. , summing the major diagonal and dividing by the total) except 
that the matrix Is normalized first. 

Smith and Itkowsky (1978) compiled five error matrices for a 
study In north central Colorado. Two of the matrices were for 
training sets; Original was compiled using a supervised classification 
while Joseslgs was compiled using a modified unsupervised classification. 
The other three matrices (Scrambll, Scrambl2, Scrambl3) were attempts 
to reclassify Incorrect pixels using a computer program called SCRAMBL. 
Table 1 shows the Joseslgs error matrix before normalization and 


Table 2 shows the matrix after normalization. 


Computer Classification Computer Classification 




Table 1. Josesigs error matrix before normalization. 


Decld. 17 


Reference Data 

Decld. Conlf. Grass Meadow Shrub Water Sage 
17 2 0 0 0 0 0 


Conlf. 28 127 


Grass 


Meadow 16 


0 1 122 


Shrub 



Water 0 

Sage 0 


0 127 


Table 2. Josesigs error matrix after normalization. 


Reference Data 


Shrub 


Decld. 

Conlf. 

Grass 

.5363 

.0996 

.1001 

.1382 

.8044 

.0158 

.0154 

.0200 

.5025 

.0879 

.0035 

.0174 

.1804 

.0180 

.0906 

.0035 

.0046 

.0230 

.0384 

m 

.2505 








Overall 

0 Performance 


T-398 


Overall 

Performanc 

4.2958 


E-4.2958 
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Table 3 shows the results of overall performance and 
normalized overall performance fir all five error matrices. Note 
that the relative accuracies are similar for the two performance 
values except for the Joseslgs matrix. Careful study of Table 1 
shows why this Is so. Only three pixels In the shrub category were 
correctly classified. This forced the normalization procedure to 
Inflate the values In the shrub row and column decreasing the 
normalized performance accuracy. Also, no sage category pixels were 
classified at all resulting In the same type of normalization 
problem. 

Table 3. Overall and normalized overall performance results 
for five classification error matrices. 

Normalized 


Matrix 

Overall Performance 

Overall Performance 

Original 

90.37Z 

86.03% 

Joseslgs 

85.96Z 

61.36% 

Scrcunbll 

85.43% 

79.97% 

ScramblZ 

78.94% 

70.49% 

Scrambl3 

80.18% 

74.17% 


i 
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Similar results were achieved for matrices compiled by Hoffer 
C1975a). Here four error matrices were compiled at two different dates l 

comparing a classification of major land cover types versus forest 
cover types. The results of normalization shown in Table 4 agree with 
tlie overall performance values calculated by Hoffer. I 

Table 4. Overall and normalized overall performance results 
for four cover type error matrices. 

Normalized 


Matrix 

Overall Performance 

Overall Performance 

Major Land 
Cover Types 
6-5-73 

85.96% 

89.51% 

Major Land 
Cover Types 
8-8-73 

69.35% 

72.53% 

Forest 

Cover 

Types 

6-5-73 

71.79% 

76.87% 

Forest 

Cover 

Types 

8-8-73 

48.83% 

57.88% 


3.2 KAPPA Analysis 

The FORTBAN computer program KAPP.A (see Appendix III) calculates 
a K statistic for a given error matrix which allows one to compare 
















error matrices to see if they are significantly different. This type 
of comparison has many uses. In an example sited above, Hoffer (1975a) com- 

A 

piled two classifications at two different dates. The K statistic and 
corresponding confidence interval (i.e., upper and lower bounds) are 
presented for each error matrix in Table 5. 

A 

Table 5. K statistic with upper and lower limits at 95% 
confidence interval for four cover type error 
matrices. 


Matrix Lower Limit K Upper Limit 


Major Land 
Cover Types 
6-5-73 

.69396 

.69458 

.69521 

Major Land 
Cover Types 
8-8-73 

.62880 

.62929 

.62978 

Forest 

Cover 

Types 

6-5-73 

.38961 

.39055 

.39150 

Forest 

Cover 

Types 

8-5-73 

. 33004 

.33074 

.33144 

















As can be seen from Table 5, none of the confidence intervals 


overlap; therefore, all these matrices are significantly different. 
This means that the Imagery taken at two different dates Is signifi- 
cantly different which Implies that one date must then be better than 
the other. A quick look at the data Indicates that 6-5-73 was the 
significantly better date. 

Another example of this technique is provided by Hoffer (1975b). 
In this example, four matrices were generated from four different 
classification algorithms. The results presented In Table 6 show 
that all the matrices are significantly different. 

A 

Table 6. K statistic with upper and lower limits at 95Z 


confidence Interval for four classification 
algorithms. 

A 

Matrix Lower Limit K Upper Limit 


Nonsupervlsed 
(10 cl.) 

.60271 

.60479 

.60686 

Nonsupervlsed 
(20 cl.) 

. 58348 

.58573 

.58799 

Modified 

Supervised 

.47326 

.47581 

.47837 

Modified 

Cluster 

.71631 

.71846 

. 72001 


A final example of the K statistic Is found in Appendix V. 
This example deals with comparing photo Interpreters to see If they 
are significantly different. 



( 

I 
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3.3 CONTABLE Analysis 

The API computer program CONTABLE (see Appendix IV) allows 
one to analyze multi-way contingency tables.* In the example here 
a 5-way table is analyzed. This table (Carneggie, 1972) deals with 5 
factors or effects listed in Table 7. The data consists of 18 ( 

5x5 error matrices with various films, dates, and interpreters. | 

5 

Table 7. List of factors and effects for 5-way contingency 1 

table. t 


FACTOR 

EFFECT 


1 

Date 

(6/10, 7/25, 10/25) 

2 

Film 

(Color, CIR) 

3 

Interpreter 

(#1, #2, m 

4 

Row 

(1, 2, 3, 4, 5) 

5 

Column 

(1, 2, 3, 4, 5) 


The hypotheses to be tested in this example are listed in 
Table 8 while the results and conclusions are listed in Table 9. 


*Without the use of this program and its Iterative Proportional 
Fitting Procedure, analysis of tables larger than 3 dimensions 
would be impossible. 
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Table 8 . List of hypothesis for CONTABLE example. 


No film effect 

No Interpreter effect 

No date effect 

No row-column effect 

No date-film Interaction 

No date- Interpreter Interaction 

No film- interpreter interaction 


Table 9. List of results and conclusions for CONTABLE 
example. 


HYPOTHESIS 

CHI SQUARE VALUE 

CONCLUSION 



^2 ’ 

0 

623.487 

reject 

«0 




0 

613.142 

rej ect 





0 

591.543 

rej ect 





- 0 

134.485 

fail 

to 

rej ect 

«0 


^2 ‘ 

0 

145.961 

fail 

to 

rej ect 



^13 " 

0 

162.393 

fail 

to 

rej ect 

«0 


W 23 - 

0 

144.707 

fail 

to 

rej ect 

«0 
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Table 9 shows Chat although no ^..agle factor significantly 
affects Che classification, the combination of two or more factors 
does. This means that none of the three factors (film, date. 
Interpreter) Is more Important Chan Che others. Instead all three 
factors Interact together to give Che best classification. From 
Che analysis so far there Is no significant one factor on which 
most of the accuracy depends. 
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4. 0 Accuracy Conference 

A National Working Conference on LandsaC Classification Accuracy 
Assessment Procedures was held In Sioux Falls, South Dakota. A * 
summary of this conference Is given In Appendix VI as a draft manu- 
script which will be revised and submitted for publication In . 
journal. 


5.0 Wildlife Habitat Assessment Methods 

A secondary task In this year's plan of work was to develop 
digital spatial analysis techniques for assessing wildlife habitat. 
Appendix VII Includes a FORTRAN computer program for doing this, and 
the techniques are described In a manuscript which Is Appendix VIII. 


6.0 Effects of Classification Accuracy on Intersperslon Maps 

Artificial land cover type maps were made In order to teat the 
effects of classification accuracy on computer generated Intersperslon 
maps. Three cellular maps were made, each containing 10 rows and 10 
columns with each cell assigned to one of 5 classes. The first map was 
used as a reference base map for comparison with the other two maps. 

The second map had 90Z of Its cells classified similar to the first 
Cl.e., 90Z accurate), and the third map was 70Z similar to the reference 
base map. Five cover types, designated 1 through 5, were used on 
each map. Similarity matrices were generated between the accurate 
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(90Z similar) "Map II" and the base map (Table 10) , and between the 
less accurate, "Map III" (70Z similar) and the base map (Table 11). 

KAPPA was used to compare the two resulting similarity matrices. 

The Intersperslon index described by Mead ^ in Appendix VIII 
was used to create intersperslon maps from each of the three fictional 
cover type maps. The maps delineate areas of high Cdesignated 3), 
medium (2), and low (.1) intersperslon. Similarity matrices were 
created by comparing each of the intersperslon maps (from the cover 
type maps II and III) with the intersperslon map made from the base 
map (Tables 12 and 13). 

The Implementation of the KAPPA program (see Section 3.2) was 
then used to test for a significant difference between the intersperslon 
maps. The resulting KHAT values indicate that cover type maps II and 
III were significantly different. A significant difference was also 
found between the two matrices for the intersperslon maps. However, 
further work is needed to understand the effect of map accuracy on 
computer generated intersperslon maps. Juxtaposition maps, and spatial 
diversity maps. Also, the effect of increasing the number of cover 
types or the number of Intersperslon classes (high, medium and low) is 


unknown 
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7.0 Summary and Future Work 

The literature review and preliminary Investigations show 
that: CD Che statistical techniques Initially proposed are sound 

and are useful for analysis of Landsat classification accuracy data, 

(2) substantial amounts of data from accuracy assessments exist 
but few sets are comparable prohibiting hypotheses from being 
tested, C3) preliminary results show that the method used In sampling 
a classification can significantly affect the estimated accuracy. 

An "automatic" computerized system needs to be developed for com- 
piling error matrices for any classification given the necessary 
ground truth and a specified sampling strategy. Experiments need 
to be designed In the future so that fundamental questions can be 
answered about factors which affect classification accuracy. 

The wildlife habitat assessment system has greatest potential 
when animals with requirements related to the spatial characteristics 
of the landscape are considered. Juxtaposition can be of great 
Importance or of very little Importance depending upon the specific 
geographic area and the wildlife species of Interest. When this data 
on the spatial characteristics of the landscape are coupled with basic 
land cover Information and ancillary data (e.g., elevation, slope, 
soil type, political or ownership boundaries), an over-all system 
for habitat assessment may be realized. Such a system could be 


Implemented on a computer and merged with data on other resource 
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attributes C®*g»» timber producing capability). Further work should 
Include pilot testing the system and an evaluation by field level 


resource managers. 
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Appendix I. 

List of Sources of Error Matrices 

1. Mead, Roy A., Landsat Digital Data Application to Forest 
Vegetation and Land-Use Classification In Minnesota. Ph.D. 
Dissertation, University of Minnesota, 1977. 

4 matrices (training set, test set, 2 evaluation areas) 
Minnesota. 

2. Fleming, Michael, Computer Aided Analysis Techniques for an 
Operational System to Map Forest Lands Utilizing Landsat MSS 
Data, LARS Technical Report 112277. 

2 matrices Colorado. 

3. Smith, James and Frank Itkowsky, Sensitivity of Variable 
Probability Sampling Estimates to Initial Landsat Classifi- 
cation, Final Report R.M.F. & R. E.S. USFS Coop-Agree. 16-741-CA, 
September 1978, CSU, Fort Collins, Colorado. 

5 matrices (training set, test set, 3 evaluation areas) 

Colorado. 

4. Madding, Robert and Harland Hogan, Detection and Mapping of 
Spruce Budworm Defoliation In Northern Wisconsin Using Digital 
Analysis of Landsat Data. Proceedings of ASP Convention. 

Feb. 26 - Mar. 4, 1978. pp 285-300. 

2 matrices (normal and collapsed) 


Wisconsin 


1 


r - 

35 

\ 

I 

5. Voss, A. W. , J. E. Bakar, G. E. Hauser, and D. W. Newton, The 
Use of Landsat Derived Land Cover Data In a Flood Peak Correia- 

■ 

tion Study, Proceedings ASP, Feb. 26-Mar. 4, 1978, pp. 135-146. 

2 matrices (normal and collapsed) 

North Carolina - Tennessee. 

6. Hoffer, Roger, Natural Resource Mapping In Mountainous Terrain 
by Computer Analysis of ERTS-1 Satellite Data, LARS Research 
Bulletin 919. Info. Note 061575. 

10 matrices (different classification systems) 

I Colorado. 

7. Hoffer, Roger, Computer-Aided Analysis of Skylab MSS Data In 

Mountainous Terrain for Land Use, Forestry, Water Resources, 
and Geologic Applications, LARS Info. Note 121275, 1975. 

: 4 matrices (varying spectral bands) 

I Colorado. 

^ 8. Hoffer, Roger, Mapping Vegetative Cover by Computer Aided 

Analysis of Satellite Data, LARS Technical Report 011178. 

2 matrices (test sites) 

Colorado. 

. 9. Hoffer, Roger, Variables In Automatic Classification over 

Extended Remote Sensing Test Sites, LARS Information Note 061571. 
1 matrix (test site) 

Indiana - Illinois. 

i 


j 
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10. Hoffer, Roger, Basic Forest Cover Mapping Using Digitized 
Remote Sensor Data and ADP Techniques, LARS Information Note 
030573. 

13 matrices (tests at different spectral bands) 

11. Heller, R. C. , R. C. Aldrich, R. S. Driscoll, R. E. Francis, 
and F. P. Weber, Evaluatldn of ERTS-1 Data for Inventory of 
Forest and Rangeland and Detection of Forest Stress. PSW & RM 
For & Range Exp. Sta. Aug. 9, 1974. 

12 matrices. 

12.. Ernst, Carola Llsette, Digital Processing of Remotely Sensed 
Dat'. .or Mapping Wetland Communities, Ph.D. Dissertation, 
Purdue University, Dec. 1979. 

6 matrices (classification) 

Indiana. 

13. Nelson, R. and R. Hoffer, Computer Aided Processing of Landsat 
MSS Data for Classification of Forest Lands, LARS Technical 
Report 102679, 1979. 

12 matrices 
Colorado. 

14. Carneggle, D. M. , Large Scale 70 mm Aerial Photographs for 
Evaluating Ecological Conditions, Vegetatlonal Changes, and 
Range Site Potential. Ph.D. Dissertation, University of 
California, Berkeley. 

18 matrices (photo Interpretation) 
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15. Lauer, Donald, Claire Hay and Andrew Benson, Quantitative 
Evaluation of Multiband Photographic Techniques, Final Report 
for Earth Observation Division Manned Spacecraft Center, 

NASA Contract NAS 9-9577, 1970. 

79 matrices (photo interpretation) 

16. Bryant, Emily and Gibb Dodge 
1 matrix. Maine. 

17. Roberts, Edwin 

1 matrix. Colorado (test set for Grand County). 

18. Roller, Norman and Larry Visser, Accuracy of Landsat Forest Cover 
Type Mapping in the Lake States Region of the U.S., Fourth 
International Symposium on Remote Sensing of Environment, 

April 23-30, 1980. 

1 matrix (Forest Cover Types). 

Michigan. 

19. Newcomer, Jeffrey 

3 matrices. Pennsylvania. 

20. Harrington, John A. and Charles W. Dunn, Jr. 

3 matrices (forest - other) Oklahoma. 
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Appendix II. Listing of FORTRAN Computer Program MARGFIT 


1 

a 

3 

4 


//'/vArPiv 

c 

c 

C 

c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 


,PAGfcSs30 

A********************************************-* 
• * 

* MAi^GFIT aiAS H(El•/KITTE^ AND i)OCUME^T£0 dy • 

* KUJiSeLL G. CONRALTON * 

* OEPr. OF FORESTRY, VPIASl * 

* JULY 1W7R * 

* * 
A********************************************* 


*•*****«•*«•«**•****«***•*«***«**********«***«*•*««*««««********•*•*« 

* 

THIS PhOGRAm was 0E5IGNE0 TO CHANGE A maTkIX CF MAXIMUM OT^ENSIONS * 
ijF 50X50 INTO A matrix rtITH PREOETERMINEO R(J« anq COLUMN MARGINALS * 

* 

NUMMaT = THE number OF MATRICES TO HE CHANREC * 

TAbII.J) = the value in RUrt I and COLUMN J UF ThE GIVEN MATRIX * 

= THE marginal value for RUW I * 

= THE MARGINAL VALUE FOR COLUMN J * 

= THE MAXIMUM NUMbER OF ITERATIONS * 

= THE MAXIMUM ALLO'AAHLE DEVIATION * 

= THE Number of rows In THE matRIx * 

= THE number of columns IN Th£ MATRIX * 

« 

A******************************************************************** 


RMAR ( I ) 
CMAR ( I J 
MA X 1 T 
MA XOEV 
IR 
IC 


5 

C 

15 

A 


50 

7 

A 

9 


10 

10 


20 

1 ! 
12 


200 

1 3 
1 a 


30 

15 
1 o 


ao 

1 7 
18 


51 

19 

20 

c 

52 

21 

22 

23 


999 

2a 

25 


53 

2b 

27 


54 

28 

29 


55 


DIMENSION TAb(50,50) ,RMAR(50) ,CMAR(SO) ,F IT(5C) 
REAL maxOEV 

NCOUNTsO 

REA0(5,15) NIJmmaT 


00 200 1=1, I R 

REA0(5,20) (TAfld, J) , jsl , ic) 


REAO(5,30) iRr'AR(i), 1 = 1, IR) 
FORMAT! 12 (Fb. on 
REAUl5,aO) (CMAR ( I) , l = l , IC) 
FORMAT(12(Fb.O) ) 

READ (5, 51) MAX IT, MAXOEV 
FUkn.AT(I5,F10.3) 

READ{b,52) 

FQkMATC'AAAAAAAAAAAAAAAAAAA 

WRI TE(b,R9R) 

FORmaTC ' 1 ') 

NRlTE(b,52) 

wRlTE(fa,53) 

F(JRmaT(Ix, '**»************• 

■vRI T£ fb, 5a) 

FOPma 1 1 1 X , ' THE original MATl 

NRlT£(b,55) 

FORMATC IX, \ 

00 250 1=1, Th 


//) 
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43 

aa 

45 

4h 

47 

4« 

4<? 

50 


53 

54 

55 
5<> 
57 
5fl 
59 
bO 
hi 
b^ 
hi 
b4 

65 

hh 

67 

bfl 

69 

70 
7 I 
72 
7 5 

74 

75 

76 

77 


/jWI TE fb.56) ( fArX t, J) , J=i , IC) 

5fa F0«MAT(12(IX,F6.J ) ) 

250 CC.NiriNUF. 

C 

NI TsO 

OQ 300 1 = 1 , I« 

FIT(I)sO.OOOOOl 
00 400 J=1 , IC 
400 FIT(I)=FIT(I)*TAB(I,J1 
300 CONTINUt 
100 1)0 150 I = l , IH 
UO 500 J=l,IC 

TAH(I,J) = TAB(I,J) *RMAR(I)/FIT(n 
500 continue 
150 continue 

no 600 J=1,IC 
d=0, 000001 
00 700 1 = 1 , IW 
7 00 H = 8-*-TAH(I, J) 

OU 600 1 = 1 , IW 

IFlTAB(I,J),LT. I .OE-1 0) TA6(I,J)=0.0 

TAg(I,J)5TA6CI,J)*CWAR(J)/6 
800 CONTINUE 
600 CONTINUE 
NIT=nIT>1 
0 = 0.0 

no 900 I = l , I« 

FIT(I)=0.000001 
00 lOOO J=1,IC 
1000 FIT(I)sFIT(n+TABfI,J) 

H = AH5(FIT(I)-«MAR(in 
IFlO.GE.rt) GO TO 900 
D=H 

90n CONTINUE 

IF (O.LE .'^AXOEV ) GO TO 2b0 
IF (NIT. LE. MAX IT) GO TO 100 
••Rl T£ (6, 60) MA X IT , u 

60 FOWMATC' no convergence AFTER*, 15,' ITERATICAS. 
A I MUM OEVIATION is: *,F10.3) 

GO TO 1200 

260 aiRITE(6,70) NIT,0 

70 FORmaTC///* convergence AFTER ',15,* ITERATIC.mS 
AIATION of: *,F10.3/) 

00 1 100 1 = 1 , IR 

wRITE(6,«0) ( tab ( I , J) , J = 1 , IC) 

60 FURMA T (20 ( 1 X ,F 7 . 4) ) 

1100 continue 

C 

1200 NCOUNTsNCOUNT+1 

IF (iNCUUNT.lt. MIJMMAT) GO TO 50 
C 

STUR 

END 


/* THE 
aTTH a 


CURRENT 


MAX I MUM 




MA X 

oev 


//Data 
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Appendix III. Listing of FORTRAN Computer Program KAPPA 


//waTFIV 


,PAGES=35 


A******************************* 


* KAPPA WAS WE-IWITTEN AND OOCUMENTfO 8Y 

* RUSSELL G. CONGALTOM 

* OEPT, OF forestry, VPI^Sl 

* JULY 1979 


!**••**•*• 


* 

THIS program was designed TO TEST FOR SIMRAR CEGREES OF AGREEMENT * 
BETWEEN Two OR MORE SQUARE ERROR MATRICES * 

* 

ME s THE NUMBER OF TABLES OR MATRICES TO BE COMPARED * 

NR = number of ROWS; ALSO THE NUMBER OF COLUMNS SINCE THE • 

MATRIX IS SQUARE * 

X(I,J) = the value in THE MATRIX FOR RQw I AND COLUMN j' * 


i***«*****«*i 


I ***** * 


REAL KHAT,LCL 

DIMENSION X(a0,a0),SXR(2O),SXC(20) 

DIMENSION UCL (20) ,LCL (20) ,KHAT (203 

Ls20 

MsO 

K=l 

REAOCS.IO) ME 
10 F0RmaT(12) 

100 DO 200 1=1, L 
SXR(n=0.0 
SXC(I)=O.U 
DO 300 J=1 ,L 
300 X(I,J)=0.0 
200 CONTINUE 

REAO(S,20) NR 
20 F0RmaT(I2) 

00 400 1=1, NR 

REA0(5,30) (X(I, J) ,J=1 ,NR) 

30 FORmaT(12(F6.0) ) 

400 CONTINUE 

REA0(5,3l) 

31 format ('AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA') i 

WR1TE(6,999) | 

999 format ( ' 1 ' ) 

WRITE (fe, 31) 

wRITE(fe,32) I 

32 FORMATCIX, '**************************************************'///) 

wRIT£(b,34) 

34 FORMAT!////, 1 X, 'THE ORIGINAL ERROR MATRIX IS:') 
aRITE(6,35) 

35 FORMATdX,' '/) 


32 


00 450 1=1, NR 

J3 


WRITE(8,38) (X(I, J) , J=l,NR) 

3a 

38 

FORMAT(20(IX,F8.0) ) 

35 

450 

r 

CONTINUE 

38 

U 

XN=0,0 

37 


DO 500 1=1, NR 

38 


DO 800 J=l,NR 

39 


SXR(I)=SXR(I)+Xf I,J) 

ao 


SXC(J)=SXC(J)+X(I, J) 

ai 

800 

CONTINUE 

U2 


XNsXN+SXR ( I ) 

43 

500 

CONTINUE 

44 


THlsO.O 

45 


TH230.0 

48 


TH3=0.0 

47 


TH4sO ,0 

48 


00 700 1=1, NR 

49 


THlsTHl+Xd, n 

50 


Th2=TH2*SXR(I)*SXC(I) 

51 


TH3=TH3*Xd, I) *(SXR(I)-*-SXC(I)) 

52 


00 800 J=l,NR 

53 


THa=TH4*X(I,J)*(SXR(I)+SXC(J))**2 

54 

800 

CONTINUE 

55 

700 

CONTINUE 

58 


THI=TH1/XN 

57 


TH2sTH2/ (XN**2) 

58 


TH3=TH3/ (XN**2) 

59 


TH4 = TH4/ ( XN<t* 3) 

80 


■^HAT(K) = (THl-TH2)/ d .-TH2) 

8 1 


S0 = SfJRT{THl*d.-THl)/(d.-TH2)**2) 


c 

c 

c 


82 


83 


84 


85 

40 

88 


87 

45 

88 


89 

50 

70 


71 


72 


73 


74 

900 

75 


78 

910 

77 


78 

920 

79 


80 

930 

81 


82 


83 

950 

84 

940 

85 


58 

«80 


A)/((l.-TH2)**3)>(l,-THn**2 •(THU-a.*TH2**2)/(l,-TH2)**4)/X.M 

THE STEPS that follow CALCULATE THE 95% CONFIOE^CE INTERVAL FOR KHAT 

UCL(K)=KHAT(K)-fl,q«,*SO 
LCL(K)=KHAT(K)-l.q6*S0 

WRITE (f>»ao) 

FORMA r (///, I X, 'lower LIMIT', 4X, 'KHAT', ux, 'UPPER LIMIT') 


wRIT£(6,«5) 

FORMAT (IX, ' , ,, ^ ' .ax. . / •‘it. ' 

wRiTE(b,50)^7TTKi ,7hi:T(K) ,urrT7) 
forma T(3X,Fa.5, 3X,F8,5, 3X,F8.5) 

K = K ♦! 

M = M 4 > I 

IF(M.LT.mE) RU to 100 
wWIT£(8,900) 

forma r (' 1 ',' SUMMARY T46LE AND CUMP AR I SONS ' ) 


/) 




wRITE(8,9lO) 

formatcix, '* 
wRI TE (8,920) 
forma T ( 1 X , 'M 
WRITE (8,930) 

FORMATdX, ' 

00 9a0 K=1 
WRITE (6, 950) K,LCL(K) ,KHAT(K) ,UCL(K) 


WRITE (8,960) 


',2x, ', 


, ttX, 


/,«X, 


.'//) 



A 


87 

88 
8<? 

90 

91 

92 

93 

94 


95 

98 

97 


9.8 

99 

100 

101 

102 
103 


//DATA 



N=ME-l 

DO 1 300 I = liiM 
no 1400 Js2,M 
IF (J.LE.I) GO TO 1400 

IF (LCLIJ) .GT.LCLH) .ANO.LCL(J) .LT.UCLdn 
IF (UCLCJ) jLT.UCL(I) .ANO.UCLC J) .GT.LCLd) ) 
rtWITE (b, 1000) I,J 
. _ . M2, » 


1000 FOkMATdX, ‘matrix 
AI2/) 

GO TO 1400 

1100 WRITEtb, 1200) 1,J 
1200 FORmaTCI X, 'matrix 
AX ',12/) 

1400 CONTINUE 
1300 continue 

iwRITE (8,1500) 

1500 FORmaU'I ') 

STOP 

END 


IS SIGNIFICANTLY DIFFERENT FROM MATRIX 


IS NOT SIGNIFICANTLY DIFFERENT FROM MATRI 


II 
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Appendix IV. Listing of APL Computer Program CONTABLE 


cn 

C23 

C33 

C43 

C53 

C63 

C73 

C83 

C93 

:i03 

C113 

C123 

C133 

C143 

C153 

:i63 

C173 

C183 

C193 

C20D 

C213 

C223 

C233 

C24] 

C25D 

C263 

[271 

C281 

C291 

C301 

C311 

C321 

C331 

C341 

C353 

C361 

C371 

C383 

C391 

[401 

[411 

[421 
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<7C0WTABI.EH0W COlV 
9 COMTOBLEHOW 

• COMTABI_E ' 

•ANALYSES OF MUI- T Z D I MENS I OHAL. CONTINGENCY TABLES' 

•S, K, LEE - BEPARTMENT OF STATISTICS, VPI SU ' 

•EMTEREI. 7/5/1976' 

I I 

' THIS PROGRAM WILL PERFORM ANALYSES OF COMPLETE OR INCOMPLETE 

•MULTIDIMENSIONAL CONTINGENCY TABLES USING A LOGLINEAR MODEL* 

• APPROACH , ' 

I I 

• DATA SHOULD BE ARRANGED INTO A CONTINGENCY TABLE, AND MAY BE 
•STORED IN AN ARRAY SMF PRIOR TO PROGRAM EXECUTION OR MAY BE • 


DATA SHOULD BE ARRANGED INTO A CONTINGENCY TABLE, AND MAY BE* 
STORED IN AN ARRAY SMF PRIOR TO PROGRAM EXECUTION OR MAY BE • 

ENTERED UPON REQUEST , WHEN ANALYZING INCOMPLETE TABLES THE INITIAL' 
FITTING TABLE MAY BE STORED SIMILARLY IN AN ARRAY NAMED ONES PRIOR' 

TO PROGRAM EXECUTION, UPON REQUEST, THE USER SHOULD ENTER A LOGLINEAR' 
MODEL WITH WHICH HE INTENDS TO FIT THE DATA, THE LOGLINEAR MODEL' 
SHOULD BE ENTERED BY THE CONFIGURATIONS AND THE DIGIT Q IS USED TO* 
SEPARATE CONFIGURATIONS, FOR EXAMPLE, THE NO-THREE -FACTOR- I NTER AC T I ON • 
MODEL OF A 3-DIMENSIONAL TABLE, (C12» C13, C23), IS ENTERED AS*' 

I 

12013023 ' 

I 

THE PROGRAM FOLLOWS AN ITERATIVE PROPORTIONAL FITTING SCHEME TO ' 
COMPUTE THE MAXIMUM LIKELIHOOD ESTIMATES OF THE EXPECTED CELL VALUES,' 
THEN THE PROGRAM COMPUTES THREE GOODNE3S-OF-F I T CHI-SQUARE STATISTICS* • 
PEARSON CHI-SQUARE, LIKELIHOOD RATIO, AND FREEMAN-TU K EY CHI-SQUAREJ' 
HOWEVER, THE DEGREE OF FREEDOM ASSOCIATED WITH THESE STATISTICS SHOULD' 
BE DETERMINED BY THE USER, AFTER ALL THE COMPUTATIONS RELATING TO THE* 
CUP.RENT MODEL ARE PERFORMED, THE PROGRAM ASKS WHETHER MORE HYPOTHESIS' 
(I,E,, ANOTHER LOGLINEAR MODEL) IS TO BE FITTED, IF NOT, PROGRAM* 
EXECUTION IS TERMINATED,' 


ALL VARIABLES ARE LOCALIZED EXCEPT SMF AND ONES, P 
EXECUTION REQUIRES THE FUNCTIONS NAMED SUM AND YESNO, " A 
PROGRAMS PLUS THIS DOCUMENTATION ARE GROUPED UNDER THE NAME 


PROGRAM ' 


EXECUTION REQUIRES THE FUNCTIONS NAMED 


ALL OF THESE' 

E CONTABLEGRP 


FOR MORE DETAILED DISCUSSION OF THE LOGLINEAR MODEL APPROACH SEE' 
THE FOLLOWING REFERENCE;' 

I 

BISHOP, Y,M,M,, FIENBERG, S,E, AND HOLLAND, P,W,, DISCRETE* 
MULTIVARIATE ANALYSIS* THEORY AND PRACTICE, CAMBRIDGE,' 
MASS,* THE MIT PRESS, 1975,' 


SAMPLE RUN OF THE PROGRAM CAN BE OBTAINED FROM 


K, LEE, 
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<7COMTAEil.E CODV 

<7 CONTA&LE J DIM ; FOLD JAJWJTJIJJfVJVV; STEFS ) IND ; M } U ; D ; C J MN I ; MD 

cn 

C2D 'THE DEFAULT PARAMETER VALUES FOR THE ITERATIVE FROF OR T I OH AL ' 
C3] 'FITTING SCHEME ARE J MAXIMUM DEVIATION = 0.01' 

C4J ' MAXIMUM NUMBER OF ITERATIONS s= 15,' 

C53 ' ■ 

C 6 J begin: 'ENTER THE DIMENSIONS OF THE TABLE' 

C7J DIM<-Q 

C31 'IS THE DATA ALREADY IN ARRAY , SMF? ( YES , NO, OR STOP)' 

C9] W4.YESN0<A<-Q) 

CIO] ->( (W = 0) » (W=l ) )/0rGO 

Cll] RESTART :' ENTER THE DATA WITH LAST SUBSCRIPT CHANGING FASTEST' 
C12] SMF <-0 

C13] <3Et;'any more data?' 

C14J W4-YESN0( Af-O) 

C15] -»( < w= 0 ) » <w= 2 ) )/ 0 »GO 

C16] 'ENTER MORE DATA' 

C17] SMF<-SMF,T<-a 
C13] -♦get 

C19] GO:->( ( 3 f/fSMF)=x/E'IM)/G 01 

C20] 'ERROR, NUMBER OF ELEMENTS INCORRECT, HERE IS THE INPUT' 

C21] gmf 

C22] 'E'lE* YOU ENTER ZERO COUNTS? LETS TRY AGAIN' 

C23] -^RESTART 
C24] GOl : SMFt-DlMfSMF 

C25] ' E'O YOU WANT TO SEE THE TABLE?' 

C26] Wf.YESNO< A<.Q) 

C27] -♦( ( w=0) f ( w=2) )/0 »goon 

C2S] GMf 

C29] goon; 'ANY fixed zeros?' 

C30] Wf.YESNO(A«-a) 

C31] -♦< (W=0) » (W=2) )/0»GOii 

C32] 'IS THE MATRIX OF ONES AND ZEROS’ ALREADY IN ARRAY ONES?' 

C33] W<-YESNO(A<-a) 

C34] -♦( ( w=0) » (W =1 ) )/0.go7 

C35] 'ENTER A MATRIX OF ONES WITH ZEROS IN THE APPROPRIATE PLACES' 

C36] GO22:oKESf.0 

C37] G06;'ONY MORE?' 

C38] Wf.YESNO< A+.Q) 

C39] -♦( < w= 0 ) f ( w= 2 ) )/0*go7 

C40] 'ENTER MORE ONES AND ZEROS' 

C41D ONES<-ONES,n 

C42] -»go 6 

C43] G 07 : A+.f ,ONES 

C44] ->( ( ( X/t«IM)=A) , ( ( X/I'IM)< A) )/GO19,GO20 

C453 'YOU HAVE NOT ENTERED ENOUGH NUMBERS YET' ALL THE REMAINING' 
C46] 'NUMBERS ARE TAKEN TO BE ONE' 

C47] ONES<-ONES, ( ( X/DIM)-A)f 1 

1 : 43 ] -♦G021 

C49] GO20J''OO HAVE ENTERED TOO MANY NUMBERS;' 

C50] G021 : ONES<-DIMf> ONES 

C51] 'HERE IS THE MATRIX OF ONES AND ZEROS AS IT STANDS NOW' 

C52] ONES 
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f • 

^ f 



C33J 

' I'a THIS THE 

C54J 

WhYESNO ( A<-a ) 

C553 

-♦( (W=iO) ♦ ( w=l ) 

C563 

'RE-ENTER YOUl 

C573 

-♦0022 

C583 

GO19;0NES<-DIMf 

C593 

' DO YOU WISH 


C603 
C613 
C623 
C633 
C64 3 
C65: 
C663 
C67] 
C683 
1691 
C703 
C713 
C72D 
C731 
C743 
C75: 
Z761 
1771 
Z7S1 
Z791 
C80: 

cai: 

C82] 

C838 

r343 

C353 

C363 

C878 

C33D 

C398 

C70] 

C913 

C92] 

C933 

C943 

[953 

C963 

C971 

C933 

C993 


MATRIX TOU WANT? 


MATRIX OF ONES AND ZEROS' 


TO SEE THE TABLE OF ZEROS AND ONES'?' 

Wf. TESNO ( A<-|]] ) 

-♦( (W* 0 ) » ( W = 2 ) )/ 0 fGO 2 
ONES 
-♦002 

GOl 1 • ONES«-DlMf 1 

G02; 'ENTER MARGINALS TO BE FIT' 

goi8:v<-q 

Ct CLE ; ' ANY MORE MARGINALS'?' 

W<-YESNO ( A<-[] ) 

-»( (W = 0> ♦ ( W = 2) )/0f G03 

'ENTER MORE MARGINALS' 

v<-v,o 

-♦CYCLE 

G03; v<-v,o 
i*-0 
Of-O 

goi2:'J«-o+1 

-♦( J>fV)/G013 

•4(vcj3=0)/go12 

vv<-, VC j] 

G014 : 

-♦( J>fV)/G016 

-♦(vcj:=0)/go17 

VV<-VV, VC J] 

-♦0014 

goi7M(vc J-13=0)/G°14 
K-i + i 
Af-f VV 

VVf-VV, ((IXfDIM)-A)fO 
-♦0014 

G013;'Y0U HAVE NOT ENTERED ANY MARGINALS YET 

-♦G018 

G016'Vf.(I,/'DIM)/»VV 
(^'YOU HAVE ENTERED 

MD<-0,01 

MNK-15 
FOLD«-ONES 
STEF5<-1 
M<-1 

'DO YOU WISH TO SET THE PARAMETER 


TRY AGAIN' 


MARGINALS . 


VALUES FOR THE ITERATIVE SCHEME'?' 


i 

C1003 

Wf-YESNO ( A«-d ) 


^ 1 

1 cioi: 

-♦( (W=0) ? 

(W = 2) )/Cf£‘ISLOOF 


C1023 

G 05 ; ' WHAT 

IS YOUR VALUE 

OF 

; 

E1033 

MDf.Q 




C1043 

'WHAT IS 

YOUR VALUE OF 

MA> 

; 

U053 

MNIf-a 




Cl >63 

BIGLOOP : -♦ 

( M > MNI ) /NOCON 


i ’ 






MAXIMUM DEVIATION'?' 


NUMBER OF ITERATIONS?' 


I 


4 ! 





y 







C1071 

C1083 

C109D 

CllOD 

Clll] 

C112D 

C1133 

C114D 

C115D 

:il63 

C117D 

[113] 

C119] 

C120] 

C121] 

[1223 

C123] 

[1243 


[1253 

nm 

C1283 

C1293 

C1303 

[1313 

C1323 

[1333 

[1343 

[1353 

[1363 

[1373 

[1383 

[1393 

[1403 

[1413 

[1423 

[1433 

[1443 

[1453 

[1463 

[1473 

[1483 

[1493 

[1503 

[1513 

[1523 

[1533 

[1543 

[1553 

[1563 

[1573 

[1583 

[1593 

[1603 

[1613 



vci ; 360 ) )/v[i ; 3 

FOUDt.FOl.riX < IMt' SUM SMF)f((IHD SUM FOLD)f(IMD SUM FOUD ) j Q ) 


H-I + 1 
+U1TUOOF 
CHECK ; I«-l 

CHECKER :.»( I > STEPS ) /OUT 

lMD«-(^(V[:i ; ]s0) )/V[I > 3 

<^<- ( [/ I f ( SUM SMF)-(IHD SUM FOLD)) 

•4(C<D)/G010 


D<-C 

<5010:->(C>MD)/STEP 

.^CHECKER 


STEP : M«-M4-1 
.tBIGLOOP 


0UT;U«-( ( , FOLDS 0)-^( rSMFsO) ) 

I I 
I I 

(Ht' MAXIMUM DEVIATION = • ) , ( f E> ) f ( r ‘ NO , OF ITERATION = *),^M 

i I 
t I 

( y • PEARSON CHI-SQUARE ' ) , f + / ( ( ( '■'^ ) / ( , SMF-FOLD ) ) * 2 ) ■?• ( ("*->)/ f F'O'-D ) 

t<-(smf*o,5) + ( (Smf+1 ) *o.5)-( < 1+4xpold ) *o . 5 ) 

( f ‘ PPEEMAN-TUKEY CHI-SQUARE ')jt+/>(T*2) 

U<-,SMFgO 

( y 'LIKELIHOOD RATIO • ) , f 2X+/ ( ( -vU ) / , SMF ) X«( ( ) / , SMF )4-( ( -vU ) / , FOLD ) 

( f • TOTAL NUMBER OF FIXED ZEROS '), ( X /t' I M )-(+/, ONES ) 

I I 
I I 

' DO I OU WANT TO SEE THE EXPECTED TABLE? ' 

Wt-T ESNO ( At-d ) 

♦< (W=0) f (W=2) )/0»CONT 


( L0.5+i^oi-Dxl00)-;-100 

I I 
I I 

•DO rou WANT TO SEE THE FREEMAN-TU K EY DEVIATIONS?' 
Wf- YESNO ( Af-Q ) 

-♦( (W = 0) » (W = 2) )/0fGO9 

I I 
I I 

< L0.5 + TxlOO)-rlOO 


G09J'D0 YOU WANT TO SEE THE RESIDUALS 
Wf YESNO ( At-d ) 

-♦( <W = 0) » < w = 2) )/0»GO4 


(OBSERVED - 


( L0.5+< smf'-f'Old) X100)-M00 



EXPECTED ) ? • 
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tl62: 

C163D 

C1643 

C1653 

C166] 

C167] 

C1683 

C169: 

C170D 

C171] 

C1722 

C173] 

C174: 

C1753 

C176D 

C1772 

C1783 

C1793 

C1803 

C1813 

C1823 

C1333 

C1843 

C1353 

C1863 

1:1373 

C1383 

C1393 

C1903 

C1913 

C1923 

1:1933 

C1943 

C1953 




I I 

G04*|£.0 YOU WrtMT TO SEE THE STAHI>HRE> 1 2E0 RES I E-U Al_ S? ' 
W<-YESNO ( A<-n) 

-♦< (W = 0 ) , ( W = 2 ) )/ 0 .GO 8 

I I 


(L0.5+< (SMF-FOLD)^( (FOL-r.+FOLD^o) *0.5) )xi00)-f-100 


GOgj'DO YOU WANT TO SEE THE LOG EKF EOT AT I OHS V ■ 

W<- YESHO ( A<-n ) 

-♦ < ( W = 0 ) f ( W = 2 ))/0f COMT 

I I 

( LO.S+C^FOLD+FOLfjO) X100)+100 

I t 
I I 

COHTJ'WAHT TO TRY SOME OTHER HYFOTHSES?* 

W4-YESHO ( A^-a ) 

-♦( (W= 0 ) . ( W =1 ) )/ 0 f G 02 

I t'O YOU WANT TO ANALYZE ANOTHER TABLE? ' 

W<-YESNO ( A<-Q ) 

-♦( (w= 0 ) » (w=l ) )/ 0 f begin i 

~»ENIi 

NOCON:(^'NO convergence after '),(fMNI),^' ITERATIONS.' 

( f ' the DEVIATIOJJ at THIS STAGE IS ' ) » 

•DO YOU WANT TO DO MORE ITERATIONS OR. CHANGE YOUR VALUE' 
'OF MAXIMUM DEVIATION?' 

W<-YESHO ( A«-(3 ) 

-M (W = 0) f ( w = l ) )/0»G05 
-»CONT 
END ; ' • 

t I 
I I 

■7 

\ 


^CONTINGENCY C03'7 
V CONTINGENCY 

'ENTER OBSERVED CELL FREQUENCIES BY ROWS' 

0 +- \0 
1-1 :o+.o,Q 

-♦LI X I ' Y ' ss ( n , ' MORE OBSERVED FREQUENCIES TO ENTER? ' ) C 1 3 

'ENTER NUMBER OF ROWS AND COLUMNS, RESPECTIVELY' 

DF<-x/(SHAPE<-Q)-l 

ROWTOT<. + /MAT<-SHAFEp O 

Nf- + /COLTOT<- + /MAT 

EKF f-ROWTOTs , xCOLTOT^N 

CHISQUARE 4 - + /, ( (MAT-EXF) *2)-i-E5-!F' 

'OBSERVED FREQUENCIES ARE* ' 

MAT 

'EXPECTED FREQUENCIES ARE* ' 

EKF 

CONTINGENCY CHI-SQUARE WITH ' ) , ( f E'F ) , < f ' DF IS; ' ) , .yCH I SOU ARE 
•7 
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I 

m 

\ 

^TWOWAY C0]'7 

•7 D TWOWAY Y ; A ; c ; K ; M j F: ; s ; s iE>e ; T ; TAE> ; TOP f TV ; V 
C13 TOPf.Q , Of □«- ' E>*TEF: TOP HEA0IMG' 

C2D SIE>E«-0>OfQ<- ' EMTEF: side heading' 

[33 t<-dc23+0xs«-dc1]+0xP:<->:'C33+0xCpdc4] 

C43 TAB<-<S,T)f OXMPl 
C53 

[63 Vf.(Yc;F:]=M)/^590 

C73 t2;tabcm; K]f.+/YCv;c] = K 

[93 -‘Ti X is7m4-m+i 
[103 

[ 1 1 3 • % TOP 

[123 

[133 A<-fTV<.' '»(9XT)f'110 

[143 TV FMTlT 
[153 

[163 SIDE 
[173 tv<-84,tv 

[183 K<-1 

[193 t3; • I • 

[203 <-r' ' ) » (fK) » (t' l')ffTV FMT tabck;3 

[213 •4T3X I s 2.K4-K + 1 

[223 

[233 apaPOP'' 

[243 (r'p’OP: this table, total responses are: ')i>t+/jTab 

7 

\ 


1 

VYESNO CD2V ■ 

7 W*.tESNO a 
[13 A^4fA,' 

[23 -»( (A/As'STOP* ) , (a/a='yes '),(a/a='no '))/gi,g2,G3 

[33 ‘TYPE YES, NO, OF: STOP PLEASE* 

C43 ■ap4f(«PD)»' 

[53 -*2 

[63 Gi:w<-o 

[73 ->0 ] 

[33 G2;wpl . ' i 

[93 -♦O 1 

[103 G3:wp2 ■ ^ 

\ ■ ^ 


L 
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cn 

C23 

C33 

ZA2 

C53 

161 

Z71 

ca: 

C?] 

C103 

Llll 

C123 

C133 

C143 

C153 

C163 

C173 

C183 

C193 

C203 

C213 

C223 

C233 

C243 

C253 

C2A3 

C273 

C283 


^THREEWftY CD3^ 

<7 W THREEWAf W 

DEF TH<-at □ , OtD<- ‘ riEF TH HEAI’ZMG' 

P:ow4-gta , otD<- ' enter row heai'Ing' 

COLUMN<-D , OtQR ' enter column HEADING' 

d<.wci3+oxf:rwC2D+OxC«.wc33+Oxs<-wC4]+oxt<-wc5]+Oxu<-wc6 3 

TAB«-(D,R,C)f 0xl<-l+0x«<-lff « 

T1 (!<c ;S] = I )/\ a; 3+oxJRl 

T 2 :=<-TC ( ’■C fT] = J)/xB; :+OxK<-l 
T3;tadci »>Jf kHR+ZSCJUIsK 
->T3X >.K«-K + 1 

->T2x I R >.u«-J + l 
-»T1X \ D2.IM + 1 

0<-Q+- ■ ' 

• • , COLUMN 


E<-f TV«- • 

TV FMTlC + OXl«-l 
(E)f 

DEPTH , ROW 
(E)f 

TVf. 134 .TV 
T4! JRl 
T5: ' 

(T- 111 111 

-♦T4X \ D >. I <- I + 1 
<E)f 

aRQROR’ ■ 

( f ' FOR THIS TADLE, 


' r (9XC)f '110 


FMT ( I , J ) ) , f TV FMT TABQIJU;] 


TOTAL RESPONSES 


) f t +/ I 
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Abstract 


A method has been developed to quantitatively test the degree 
of similarity between photo interpreters. This method involves giving 
each photo interpreter the same set of photos to interpret. An error 
matrix is then .generated for each interpreter by comparing his interpre- 
tation to the actual ground cover. This error matrix is then analyzed 
using a computer program called KAPPA. This program uses discrete multi- 
variate analysis techniques to determine if one error matrix (i.e., photo 



interpreter) is significantly different from another. The program 


Not only does this technique allow one to compare two separate in- 


terpreters, but it also allows one to test whether an individual photo 


interpreter is consistent through time 


Introduction 


Photo interpretation is the art and science of identifying 


objects and deducing their significance on aerial photos. Good, con- 


sistent photo interpretation depends upon the experience and skill of 


the individual who delineates the boundaries between vegetation/ land 


cover types over the landscape. The judgment involved is generally 


qualitative in nature, and therefore difficult to evaluate or compare 


with interpretations made by others. Usually the interpreter has 


intuitive feelings about how well he is doing, but is unable to support 


these feelings with any specific tests. This paper suggests a way of 


quantifying photo interpretation results and gives a statistical method 


The procedure proposed in this paper can test for the degree of 


useful when more than one interpreter is to work on a project. If it can 


I 


52 




significantly different, then the project will yield uniform results 
for all interpreters. Also, it would be useful to test the same 
Interpreter over a period of time to check for changes in his inter- 
pretation. It may also be important to determine if varying types of 
photography (film/filter combinations), or seasons of photography result 
in significantly different delineations. By placing a grid over each 
delineation, the individual cells are assigned to the land cover/vegetation 
type which represents the majority of the cell. Each cell is then com- 
pared one-by-one with the corresponding cell (i.e., in the same location) 
from another delineation. If one of the delineations is assumed to be 
correct (reference data) , then comparison of the two sets of spatially 
defined cells yields a measure of "photointerpretation accuracy". This 
is usually expressed in Che form of an error matrix. 




I.) 
■ ^ 


Procedure 


An error matrix is a square array of numbers set out in rows and 
columns which expresses Che number of cells assigned by Che photo interpreter 
to a particular land cover type relative to the actual land cover (reference 
data). The columns represent the reference data and Che rows indicate 
Che photo interpreter assigned land cover type (Figure 1). 

Reference Data 
ABC 

Photo ^ 

Interprec- 
taCion 

C 



Figure 1. Error matrix format for three land cover types. 
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The numbers In the error matrix are tallies compiled by com- 
paring the photo Interpretation with the actual cover type (reference 
data) on a cell by cell basis. All correct classifications are 
located on the major diagonal of the error matrix. 

The specific method used to generate an error matrix Is dependent 

on what information is needed. If the degree of similarity between 

two or more photo Interpreters Is to be determined, each Interpreter 

Is given the same aerial photographs to Interpret. An error matrix 

is then tabulated for each interpreter by comparing his interpretation 

with a reference data set (correct delineation). If the test Involves 

determining the consistency over time for a single interpreter, then a 

representative part of a selected stereo pair is interpreted at the j 

! 

beginning of a project. At some later date the remainder of the photos ! 

are interpreted and then the two error matrices (Time A and Time B) 
are compared. Finally, if it is desired to measure the accuracy of 
delineations made on different types of photography, a separate inter- 
pretation is performed on the same area for each set of photos by each 
interpreter and an error matrix is generated. 

Once the error matrices are generated, a discrete multivariate 
analysis procedure (Bishop et , 1975) is used to test the degree of 
similarity between the error matrices. This test is based on a maximum 
likelihood estimate of the multinomial distribution (Equation 1). 
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r 
z X 
i-l 


ii 






( 1 ) 


where: 

K “ of rows in matrix 

“ If of obs in row i and col. i 
X^_^ » marginal total of row i 
X_^^ ■ marginal total of col. i 
N ■ total If of observations 

This equation yields a value KHAT which is a measure of the actual 
agreement minus the chance agreement. A confidence interval at a given 
a-level is then placed around the value of KHAT calculated for each 
error matrix. If the confidence interval for one error matrix overlaps 
the confidence interval for another error matrix, the two matrices are 
said to be not significantly different at that a-level. However, if no 
overlap of the KHAT confidence intervals occurs, then the matrices are 
said to be significantly different at that a-level. 

This entire comparison process can be performed using a FORTRAN 
computer program called KAPPA. Given the error matrices to be analyzed, 
the program calculates a KHAT value and a confidence interval for each 
error matrix. The program then prints out which error matrices are 
significantly different and which are not. 
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The daca used In this study were taken from Lauer £t al . 

(1970). Five photo Interpreters Interpreted the same aerial photo- 
graphs of Yosemlte Valley, California, and their Individual error 
matrices were generated. Also, five film and filter combinations 
were used with a single Interpreter, and error matrices were generated. 

Results and Discussion 

i All five of the Interpreters tested on the photos from Yosemlte 

i 

j Valley produced significantly different delineations (Table 1). The 

I confidence Interval was calculated at the 95Z level. 

Table 1. Summary table for five Interpreters of Yosemlte Valley 
! - photos. 


1 

Interpreter 

Lower Limit 

KHAT 

Upper Limit 

i 

i 

i 

1 

1 

0.31167 

0.31991 

0.32815 

2 

0.28623 

0.29420 

0.30216 

1 

1 

3 

0.36677 

0.37485 

0.38293 

i 

4 

0.23115 

0.24156 

0.25197 

1 

1 

i 

1 

5 

0.20878 

0.21925 

0.22972 

The results 

of the five different 

film and 

filter combinations 


are presented In Table 2. These results were also calculated at the 
95Z confidence level. 
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Table 2. Summary table for the five film and filter combinations. 


Film/Pilter 

Lower Limit 

KHAT 

Upper Limit 

IR-301/W25 

0.31167 

0.31991 

0.32815 

1P/W89B 

0.29615 

0.30436 

0.31258 

Ekta Aero IR 

0.11318 

0.12071 

0.12825 

Enhancement X 

0.25427 

0.26163 

0.26898 

Enhancement Y 

0.36704 

0.37438 

0.38173 


As can be seen from Table 2, the Interval for IR-301/W25 over- 
laps with the Interval for IR/W89B. Therefore, these two Interpre- 
tations are not significantly different. All the other Interpretations 
are significantly different. 


Summary 


The examples given In this paper Indicate how photo Interpre- 
tation results can be quantified using error matrices. These error 
matrices can then be compared using a discrete multivariate analysis 
procedure and conclusions made. 
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■ ABSTRACT 

A working conference was held in Sioux Falls, South Dakota November 12, 
13, and 14, 1980 dealing with Landsat Classification Accuracy Assessment 
Procedures. Thirteen formal presentations were made on three general topics: 
(1) sampling procedures, (2) statistical analysis techniques, and (3) examples 
of projects which included accuracy assessment and the associated costs, 
logistical problems and value of the accuracy data to the remote sensing 
specialist and the resource manager. Nearly twenty conference attendees 
participated in two discussion session addressing various issues associated 
with accuracy assessment. This paper presents an account of the accomplish- 


ments of the conference. 


INTRODUCTION 


In th.e years since Landsat imagery first became available, an untold 
number of Landsat scenes have been digitally analyzed to classify land cover. 
These classifications are not without error, and have been subject to closer 
scrutiny by critics and potential users than similar products developed by 
more traditional methods. A few potential users of Landsat data were discouraged 
by the unfulfilled expectations spirited by the results of early investiga- 
tions. This has recently led researchers and government agencies to proceed 
cautiously with technology transfer. Thus, scientists have been keenly aware 
of the need to assess the accuracy of Landsat classifications before dis- 
tributing the products to users. 

Topographic mapping procedures include routine evaluations for compliance 
with well defined accuracy standards and the accuracy attainable under specific 
conditions (terrain characteristics and mapping equipment used) are well 
known. This capability is the result of many directed research efforts. 

However, techniques for assessing the accuracy of Landsat classifications have 
developed in an ad hoc manner. Many such methods are not statistically sound 
and can yield biased estimates of accuracy. 

For example, researchers used the limited available ground information 
(i.'e., maps, photo interpretations or less often actual visits to the field) 
collected for development of training statistics to estimate classification 
accuracy. This can result in over optimistic estimates of classification 
performance, particularly when the training data does not adequately describe 
the scene variability. Windshield surveys, in which a few easily accessible 
areas are visited on the ground, are another biased approach to accuracy 
assessment. In addition, biases can also be introduced by using a different 
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classificatioa framework for accuracy assessment than that used in developing 
the digital classification. 

The trend, more recently, has beeh to sample the classifications and measure 
the degree of agreement with a set of spatially defined reference data (i.e., 
ground truth). Analysis of the resulting accuracy data can guide researchers 
in scene selection (season, etc.), and determine the most appropriate methods 
of classification for particular applications. 

The importance of assessing classification accuracy, the lack of any 
standard procedures, and the limited number of reports in technically reviewed 
journals, justified the conference discussed in this paper. Only a relatively 
small number of researchers have worked in the subject area to any great 
extent. Therefore, attendance at the conference was limited and by invitation 
only. The specific objectives of the conference were: 

1. To determine the state-of-the-art for accuracy assessment procedures. 

2. To provide a forum for exchange of ideas concerning accuracy assess- 
ment procedures. 

3. To identify research needs and recommend the approach that should 
be taken to improve accuracy assessment procedures. 

CONFERENCE THEMES 

A comprehensive proceedings of the formal conference presentations is 
planned. However, it is worthwhile to identify and summarize the major themes 
that developed from the conference in general. 

Accuracy is a measure of the amount of agreement between two data sets. 
Typically this is a thematic map in question and a reference data set often 
thought of as "ground truth." However, when this procedure is generalized 
other applications become apparent, including change detection analysis for 
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the moaitoring of particular resources. Furthermore, sequential appraisal of 
a classif cation can result in better end results. 

There are several types of accuracy and it is important to identify which 
is being utilized. Two major categories of accuracy are site specific and non 
site specific accuracy. Non site specific accuracy compares tabular summarys 
of the proportions of the area mapped into each of the categories. Site specific 
accuracy utilizes the spatial nature of the data. That is, two spatially 
defined data sets are registered and compared for the amount of agreement. 

This can be a polygon, grid cell, or point comparison. In this case, the 
difference between the two data sets results in a spatially defined binary 
data set. This represents the population we are sampling for the parameters 
in question. 

An error matrix or contingency analysis approach to accuracy assessment 
is still another method of comparison of the two data seta. This requires a 
site specific (spatially defined) approach. 

Furthermore, many factors affect the validity of an accuracy assessment. 

The quantity and quality of ground truth depend upon the methods used for 
sample size determination and data acqusition. In light of this, it becomes 
apparent that the term "ground truth" is ill defined. What is "ground truth" 

i 

with regard to parameters such as percent of ground cover? Can this ever } 

really be measured? For many cover types, this parameter can be estimated 

more accurately on aerial photographs than by ground procedures. I 

Finally, one should not lose track of the difference between the use- 
fulness of a specific product and its estimated accuracy. A numerical report 
of product accuracy may say nothing of how much use the product gets or how 
well it compares with what was previously available. A quantitative accuracy 
assessment results in a numerical summary which may or may not represent the 
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usefulness of the product. In many instances, a classification of low or 
intermediate accuracy is a welcome and useful product. 

The desired information as well as the nature of the scene which was 
classified, determine which is the most appropriate means of assessing accuracy. 
Certainly, different landscapes may need to be sampled differently for best 
results. Therefore, studies should be done to look at the sensitivity of 
accuracy estimates when different sampling procedures are used. In comparing 
and assessing sampling procedures for accuracy assessment, not only is statistical 
variability to be considered, but also the spatial diversity of the data. 
Furthermore, all of these considerations interact to determine the most appropriate 
sampling and estimation procedure to use. Much work remains to be done, 
utilizing designed experiments with specific hypotheses, to identify the 
relative reliability of various sampling procedures. 

Assessing and reporting, by some standard means, the accuracy of a thematic 
classification will become more vital as these products become a part of 
geographic information systems. This will be necessary to insure high quality 
output products and well informed management decisions. 

The use of training data for accuracy assessment results in a somewhat 
biased but possible useful estimate of overall accuracy. The nature of the 
bias is to overestimate accuracy. The amount of bias depends upon how well 
the training data represent the variability present in the scene. In some 
instances, such an approach will be adequate. However, for close scrutiny and 
for within class estimates of accuracy, and independent accuracy assessment is 
warranted. An approach to minimizing the cost of an independent accuracy 
assessment is to collect accuracy assessment data at the time the training 
data is collected. This data should be earmarked for later use and not used in 


the training process. 
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Although this coaference did much to establish coamunicatioa amoag research- 
ers utilizing accuracy assessment procedures, much work remains to be done in 
summarizing what procedures are most commonly utilized. In addition, a bibliography 
of the literature and available computer programs should be compiled and 
published. A survey of researchers in the field will help to define how well 
they can map various cover types. This will assist in developing a set of 
mapping standards. Although accuracy requirements may vary among cover types, 
acceptable map accuracy standards are needed to match intended uses. Standards 
such as "second order at level II” can help in minimizing subjective evaluations 
and finally, perhaps many classifications are more accurate than we think due 
to geometry problems and edge pixels. It becomes apparent that classification 
error and mapping error are not one in the same. Much work needs to be done 
to discriminate between the two sources of error. 
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SUMMARY 

Many issues were discussed aad debated by the participants. Topics for 
further research were identified and major themes summarized in this paper. 

The participants recommended that a working group be established to write 
a "manual" or "guide book" on accuracy assessment procedures. Possibly this 
group could be formed as an ad hoc committee within the American Society of 
Photogrammetry and seek funding to prepare the document described above. 

Plans are now being made to do this. 

The conference succeeded in accomplishing the three objectives stated 
earlier. A comprehensive proceedings is planned which will represent state-of* 

i 

the-art accuracy assessment procedures. 
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Appendix VII. Listing of FORTRAN Computer Program WILDINT2 
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Appendix VIII. 

A COMPUTERIZED SPATIAL ANALYSIS SYSTEM 
FOR ASSESSING WILDLIFE HABITAT 
FROM VEGETATION MAPS* 

I • Roy A. Mead, Terry L. Sharik, 

i • 

Stephen P. Prisley, and Joel T. Heinen 
I Department of Forestry 

Virginia Polytechnic Institute 
and State University 
Blacksburg, VA 24061 

I 

ABSTRACT 

Vegetation and land cover patterns affect the quality of habitat available 
for wildlife. Given the degree of interspersion of cover types and relative 
value of each edge type and the importance of spatial diversity, an index of 
habitat spatial diversity can be computed for each parcel of land (of 
. any desired size) relative to each wildlife species or group of species. 

T This is accomplished by defining a grid which is either placed on a land 

cover or on an aerial photograph. Each cell is then coded on the basis 
. of (its predominant) cover type. A computer program subsequently analyzes 

i the arrangement of these coded cells and produces naps of (a) interspersion, 

i 

I * Presented at 47th Annual Meeting .^erican Society Photo gramme try, Washington, 
D.C., February 23-26, 1981. 
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(b) JuxCoposltlon, and (c) spatial diversity. Separate multicolor maps 
can be made for any wildlife habitat of interest using a digital film 
recorder. These map overlays can be used by the resource manager to 
compare wildlife habitat quality and potential with maps for forest, range, 
watershed and recreation potential. 

INTRODUCTION 

There is a tremendous need tc develop quantitative methods to assess 
wildlife habitat. This was specifically mandated by the Resources Planning 
Act, as well as ocher legislation. Wildlife habitat must be considered in 
all management plans together with timber, range, recreation and watershed. 
While timber inventories have been conducted for many years, techniques for 
quantifying the wildlife habitat still need to be developed. 

The technology of remote sensing has provided the means for mapping land 
cover/vegetation over very large areas for wildlife habitat management 
(Pengelly, 1978). However, the maps themselves only partially fulfill Che 
inventory data needed by biologists who must manage for wildlife. The 
maps must be analyzed and interpreted Co enhance the various characteristics 
of Che landscape which have a bearing on management decisions. In short, 
the standard land cover map is a source of information that may be helpful 
in making management decisions. 

This paper suggests a means to analyze and interpret maps of land cover 
CO produce spatially defined data that will be valuable information for 
managing wildlife habitat. Emphasis is on the technique and not on the 
controversial issue of defining habitat QUALITY. It must be understood 
chat Che landscape characteristics important in habitat evaluation vary 



according to region ai.d the specific wildlife species of interest. The 
various weighting factors discussed in this paper must be determined by 
wildlife managers familiar with local conditions or from agency handbooks 
which give the habitat requirements and preferences for many species. 


The specific objective of this study was to develop a computerized 
system for measuring the spatial diversity component of wildlife habitat 


Study Area and Input Data 


The area used as an example for testing the wildlife habitat analysis 


techniques described in this paper was the Great Dismal Swamp. The area 


is managed by the U. S. Fish and Wildlife Service as a game refuge and 


includes appro.ximately 84,900 hectares. This wetland was thoroughly 


described by Garrett and Carter (1977) . The area was ideal for evaluating 


the proposed habitat analysis techniques for three reasons. First 


the Dismal Swamp "contains a remarkable diversity of vegetative communities 


(Garrett and Carter, 1977) . Second, the area had recently been mapped 


(Gammon and Carter, 1979). Third, the local resource managers were 


available for assistance in evaluating the validity/usefulness of the 


final habitat quality maps that were produced 


The vegetation maps produced by Gammon and Carter (1979) contained "43 


separate canopy designations and 243 specific vegetative communities 
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This map was overlaid with a square grid system oriented in a North-South 
manner. Each cell contained 22 hectares and formed a matrix of 93 rows 
and 42 columns. Each cell was given a community designation according to 
the cover type which occupied the most area within that cell. This informa- 
tion was stored on discs for analysis by the computer. 





Given the above data the following procedure was used to assess wildlife 
habitat diversity for the Great Dismal Swamp. Since the primary thrust 
of this paper is to present a proposed technique, all additional inputs 
(e.g., juxtaposition weighting factors and restrictive factors) are purely 
hypothetical, as is the selected wildlife species "A". 

HABITAT ASSESSMENT PROCEDURE 

There are four components that form the package of techniques used for 
assessing wildlife habitat; 

1. Input data 

2. Measurement of interspersion 

3. Measurement of juxtaposition 

4. Recognition of exclusion factors 

Basically, the four components interact in the following way. Suitable 
land cover /vegetation maps are either obtained from existing sources or 
compiled. The necessary vegetation categories, map scales and minimum 
mapping unit size may vary from region to region and with the species for 
which potential h'.bitat is being assessed. The habitat criteria for the 
species of interest must be known (or estimated) . Such criteria include 
the relative desirability (i.e., the weighting factors) of various 
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vegetation/ land cover edges and the animals' preference for various 
vegetation distribution patterns. Classification of vegetation groups is 
sometimes rather arbitrary (Pielou, 1977) , and must be made biologically 
in terms of the requirements of the organism? involved. For example, the 
timber type classification system used by forest Industries may not always 
be adequate for use in wildlife habitat inventories. Finally, specific 
restrictive factors or resources (e.g., water) that either MUST or 
MUST NOT be present for suitable habitat need to be known. 

A spatial diversity "SD" index value is computed for each parcel of land 
(cell) (of any desired size) relative to each wildlife species or groups 
of species. The index is a function of "IS," interspersion, "JX," 
juxtaposition, and any number of restrictive factors. 







n 


"IS", . , "JX", 

^'^A 8 ^ ^ ^“a 12 > 


M 

1 

a 

A 
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where : 

A -*• indicates a specific wildlife species or group of species (B, 

C, D, etc. for others). 

o ■* indicates the relative importance of intarspersion to juxtaposition 
for wildlife species A, 3, or C, etc. 

a indicates the relative importance of juxtaposition to interspersion 
for wildlife species A, 3, or C, etc. 
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Noce Chat a and a can range between 0 and +1 but must sum Co 1.00. A low 
value would indicate a very undesirable or unimportant characteristic and 
+1 as very desirable or important characteristic. Scaling will have to 
be worked out and a sensitivity analysis performed. 

I 1 [ -► indicates a ’restrictive factor that is essential for wildlife species 
A 

group "A”. An example of a restrictive factor might be the presence of 
water within one mile. If this is present (i.e., satisfying a necessity), 
then I 1 1 is given a value of 1 and has no impact on Che value of IS^. 

is assigned 
A 

a value of "0” and automatically makes IS^ *0. In some cases Che 

restrictive factors ma-' be set at intermediate values indicating undesirable 
conditions but not total exclusion. Values fo IS^ referring to "high," 
"medium," and "low" have to be determined (cat ^orized) . 

Necessary Land Cover Data 

It is assumed chat a suitable vegetation map is available w'nich includes the 
necessary categories of overscory and/or understory communities indicated. 
This must be determined for each wildlife species for which habitat is to 
be assessed. 

A. Small Area, Manual Analysis . A grid drawn on clear plastic material 
is placed directly over the vegetation map. The predominant vegetation 
category in each cell of Che grid is determined, and coded directly on the 
clear plastic using a grease pencil. key will be needed to relate the 


However, if there is no water (an absolute necessity), then E 





letters or symbols used to the vegetation categories. 

B. Large Area, Computerized Analysis . A vegetation map in polygon 
form is digitized (or aanually coded) at any desired cell size. Individual 
cells are categorized and a file created to store the resulting data. 

Measurement of Intersnerslon 


A. Small Area, Manual Analysis . The vegetation category/ predominant 
in each individual cell on the clear plastic grid is compared to each of 
the immediately adjacent cells. The number of adjacent cells of another 
vegetation type are counted and that number recorded on the plastic sheet 
in the lower right hand comer. 

Consider the following two examples: 


E:camnle I 


Example II 



c 

B 


B 

c 

C 



IS - 3 


IS - 7 


The center cell in example 1 has 3 adjacent cells with dissimilar predominant 
vegetation types. Therefore, the value for interspersion is 3. In the 
second example, the IS value is 7. It is clear chat the land cover 
patterns are much more intermixed in example II. Those cells with IS 
values of 7 or 3 could be printed light gray, values of 3-6, intermediate 
gray, and 0-2 as dark gray. Note chat each cell in Che entire matrix 
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becomes the centroid cell for comparison with adjacent cells. Thus, a map 
of interspersion is produced from all of the "IS" values computed by moving 
the 3x3 matrix throughout the data set. 

3. Large Area, Computerized Analysis . A computer could easily be 
programmed to compare adjacent cells and create a file with the interspersion 
values for each cell. Any range of IS values could be assigned a specific 
color or gray tone, and thus an interspersion map could be made. 

Measure of Juxtaposition 

Wildlife biologists know that certain types of vegetation edges are very 
Important for specific wildlife species. Abundance of these species 
may be considered a consequence of edges where types of food and cover come 
together (Leopold, 1936). According to Odum (1971), the edge effect may 
be defined as the tendency for an increase in variety and density of 
organisms at community junctions. This effect is most marked in animals 
with relatively low mobility (Leopold, 1936) and high requirements in 
terms of diversity of vegetative communities (Leopold, 1936). Various 
edge combinations can be assigned a relative weighting factor for each group 
of wildlife, e.g., 

A/B .60 

A/C .30 

B/C .10 

In this case the relative value of an A/3 edge is twice that of an A/C 
edge for a particiular wildlife species. Therefore, a measure of juxta- 
position can be easily computed by summing Che various quantity-quality 




produces for all edges relative to each centroid cell in the data matrix. 
Considering example matrices I and II again: 



Example II 




The JX value for example I is 2.40 and 4.50 in example II which has more 
edges which are of importance to the wildlife species under consideration. 

* Note that diagonal edges only count 1 while either vertical or horizontal 



edges count as 2. 
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RESULTS AND DISCUSSION 


A portion of the original coded vegetation map and the resulting maps for 
Intersperslon, juxtaposition and spatial diversity are shovn In Figures 
1, 2, 3, and 4. The area shown Includes 20 rows and 28 column^ of the coded 
Input data. The numbers In: Figure 1 correspond to coefficients which were 
arbitrarily assigned to the various vegetation categories mapped by 
Gammon and Carter (1979).. 


The dark. Intermediate gray and light areas In Figure 2 represent low, 
medium and high Intersperslon, respectively. These correspond to the 
following ranges for the "IS" calculation, respectively: 


0 to .3 


>.3 to .6 


>.6 to 1.0 


The designations of dark. Intermediate gray, and light in Figure 3 show 
juxtaposition and correspond to these ranges for the "JX" calculation, 
respectively: 


0 to .3 


> . 3 to .6 


>.6 to 1.0 


Finally, the spatial diversity index "SD" was categorized in an identical 
way. The resulting map is shown in Figure 4. 
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SUMMARY AND CONCLUSIONS 

A wildlife habicac diversity map was produced for a hypothetical wildlife 
species in the Dismal Swamp utilizing a vegetation cover map. This method 
can be performed very quickly by computer over large areas, given the 
necessary input data. Maps of interspersion and juxtaposition can be 
produced as well by assigning printer symbols to arbitrarily designated 
categories for each of the three parameters (interspersion, juxtaposition 
and the wildlife habitat diversity index) . Such maps are repeatable and 
would be consistent over large areas. The most crucial part of the 
operation is the assignment of the weighting factors from "known" ecological 
information about each wildlife species. The computerized methodology 
may have tremendous potential when implemented with remotely sensed digital 
data for land/cover vegetation. 

Further work is needed to determine the sensitivity of the output maps to 
changes in the weighting factors for various species of wildlife. The 
relation between animal home range and suitable cell size must also be 
examined. More efficient methods should be used to digitize the land 
cover/vegetation maps. Finally, the maps must be more thoroughly evaluated 
by field resource managers and wildlife habitat specialists. 

The method proposed here measures only the spatial diversity of the landscape. 
Such a measure, and the maps which result, could be incorporated into a 
larger, more comprehensive system for assessing wildlife habitat quality. 
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Figure 2. Interspersion map for Che same area shown in Figure 1 
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Figure 4. Spatial diversity map for the same area shown in Figure 1. 

"A" is low quality ("I" is 0 to .3), "B" is intermediate 
("I" is >.3 to .6) and "C" is high quality ("I" is >.6 to 1.0). 
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